Schema-based spatial partitioning in a time-series database

ABSTRACT

Methods, systems, and computer-readable media for schema-based spatial partitioning in a time-series database are disclosed. A time-series database divides elements of time-series data of a plurality of time series into partitions according to a clustering scheme. The time series are associated with respective schemas. The schemas comprise dimension names and measure names. The clustering scheme is determined based (at least in part) on the schemas and dimension values, and the schemas are determined based (at least in part) on the elements of time-series data. The time-series database stores the elements of time-series data from the plurality of partitions into one or more storage tiers. The time-series database performs a query of the time-series data in one or more of the storage tiers. The query is performed based (at least in part) on the clustering scheme.

This application is a continuation of U.S. patent application Ser. No.16/579,715, filed Sep. 23, 2019, which is hereby incorporated byreference herein in its entirety.

BACKGROUND

Many companies and other organizations operate computer networks thatinterconnect numerous computing systems to support their operations,such as with the computing systems being co-located (e.g., as part of alocal network) or instead located in multiple distinct geographicallocations (e.g., connected via one or more private or publicintermediate networks). For example, distributed systems housingsignificant numbers of interconnected computing systems have becomecommonplace. Such distributed systems may provide back-end services orsystems that interact with clients. For example, such distributedsystems may provide database systems to clients. As the scale and scopeof database systems have increased, the tasks of provisioning,administering, and managing system resources have become increasinglycomplicated. For example, the costs to search, analyze, and otherwisemanage data sets can increase with the size and scale of the data sets.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example system environment for schema-basedspatial partitioning in a time-series database, according to someembodiments.

FIG. 2 is a flowchart illustrating a method for schema-based spatialpartitioning in a time-series database, according to some embodiments.

FIG. 3A, FIG. 3B, and FIG. 3C examples of hierarchical clustering ofingested time-series data according to derived schemas, according tosome embodiments.

FIG. 4 illustrates an example of storage of time-series data usingtwo-dimensional tiles in a hot tier, according to some embodiments.

FIG. 5 illustrates the storage of time-series data along with schemametadata using a set of key-value data stores, according to someembodiments.

FIG. 6 illustrates the querying of time-series data using schemametadata co-located with the time-series data, according to someembodiments.

FIG. 7 illustrates further aspects of the example system environment forschema-based spatial partitioning in a time-series database, includingthe use of a user-specified ordering of schema components forclustering, according to some embodiments.

FIG. 8 illustrates an example computing device that may be used in someembodiments.

While embodiments are described herein by way of example for severalembodiments and illustrative drawings, those skilled in the art willrecognize that embodiments are not limited to the embodiments ordrawings described. It should be understood, that the drawings anddetailed description thereto are not intended to limit embodiments tothe particular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope as defined by the appended claims. The headings usedherein are for organizational purposes only and are not meant to be usedto limit the scope of the description or the claims. As used throughoutthis application, the word “may” is used in a permissive sense (i.e.,meaning “having the potential to”), rather than the mandatory sense(i.e., meaning “must”). Similarly, the words “include,” “including,” and“includes” mean “including, but not limited to.”

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments of methods, systems, and computer-readable media forschema-based spatial partitioning in a time-series database aredescribed. A time-series database may include a set of ingestion routersthat receive and spatially partition time-series data into a set ofnon-overlapping partitions. A particular time series may be defined by aschema that includes components such as one or more dimension names anda measure name. As ingested by the time-series database, an element oftime-series data may include all the schema components as well as one ormore dimension values and a measure value that represents a particulardata point in the time series. The time-series database may also includea set of stream processors, also referred to as writers, which processthe time-series data in the partitions. For example, the streamprocessors may write elements of time-series data to one or more storagetiers, such as a “hot” tier that offers low-latency andhigh-availability write and read access to a set of distributed storagenodes. The time-series database may further include a set of queryprocessors that perform queries of the time-series data in the one ormore storage tiers.

To optimize tasks such as queries, datasets may be clustered intopartitions using a clustering scheme that co-locates related timeseries. The clustering scheme may represent a multi-level hashingscheme. The clustering scheme may be based (at least in part) on schemasfor individual time series that are derived from ingested data and notnecessarily formally specified by customers. The clustering scheme mayalso be based (at least in part) on dimension values. For example, togenerate a hash representing an individual time series, a clusteringscheme for a particular table may first hash the measure name for thevarious time series, then concatenate a hash for the dimension names,then concatenate a hash for the dimension values. Using this particulartechnique, by representing the measure name as a prefix in the hash foran individual time series, various time series that have the samemeasure name may be clustered together in the partitions and in thestorage tier(s). Queries for time-series data having the same measurename may then be performed more efficiently, e.g., by identifying thepartition(s) associated with the hash of the measure name andimplementing the query only using the identified partition(s) in therelevant storage tier(s). Different clustering schemes may use differentorders and interleaving approaches for the dimension names, dimensionvalues, or measure names. In one embodiment, customers may indicateorderings of schema components for particular time series, e.g., tooptimize storage and retrieval of time-series data for the needs ofparticular customers. However, in the absence of such customer-providedinformation, a default clustering scheme may be used to partitiontime-series data such that the majority of queries are optimized.

The time series for a particular customer may have a high cardinality.For example, a set of Internet of Things (IoT) sensor devices managed bya customer may emit sensor data over time for a large number oflocations (e.g., dimension values for a location dimension), such thateach individual device and measurement type is represented by adifferent time series (and corresponding schema with individual values).A customer may seek to explore the time-series schemas and dimensionvalues under its management (e.g., in a particular table of thetime-series database) to generate appropriate queries using some portionof the schemas (and dimension values), perform validation of theschemas, or perform other tasks that require knowledge of the schemas.Instead of maintaining a separate metadata index for schema information,the schema information (including dimension values) may be co-locatedwith the corresponding time-series data, e.g., in the same tiles in ahot storage tier or files in a cold storage tier, such that the schemametadata can take advantage of spatial and temporal scaling capabilitiesof the time-series database. The co-location of time-series data withschema metadata may provide for quick discovery of schemas in order tooptimize schema exploration as well as queries.

As one skilled in the art will appreciate in light of this disclosure,embodiments may be capable of achieving certain technical advantages,including some or all of the following: (1) reducing the latency ofqueries in a distributed time-series database by using a clusteringscheme to co-locate related time series based on their schemas; (2)improving the performance of queries in a distributed time-seriesdatabase by providing low-latency exploration of schema metadata usablefor building or revising queries; (3) reducing storage use in adistributed time-series database by co-locating time-series data andmetadata instead of maintaining them using separate resources; (4)reducing storage use in a distributed time-series database by using aclustering scheme to co-locate related time series for more efficientcompression; (5) improving the performance of schema validation in adistributed time-series database by using a clustering scheme toco-locate related time series based on their schemas; (6) improving thescalability of a time-series database for high-cardinality data setsusing partitioning and auto-scaling for both time-series data andmetadata; and so on.

FIG. 1 illustrates an example system environment for schema-basedspatial partitioning in a time-series database, according to someembodiments. A distributed time-series database 100 may ingest and storetime-series data 191 and make the stored data available for queries andother computations and tasks. Elements of the time-series data 191 maybe received by the database 100 from clients 190 over time, e.g., as oneor more streams of time-series data. Clients 190 may represent varioustypes of client devices that generate or otherwise provide data invarious time series to the database 100. A time series may include a setof values that change over time, such as sensor measurements or systemmetrics, and that are timestamped or otherwise positioned along atemporal axis. For example, a set of client devices 190 may repeatedlygather information such as vibration, temperature, and pressure usingsensors. As another example, a set of client devices 190 may detectstate transitions, e.g., in a computer network. Client devices 190 thatprovide the time-series data 191 to the database 100 may be associatedwith various domains such as Internet of Things (IoT) and “smart home”networks, autonomous vehicles, manufacturing facilities, distributionfacilities, computational resources in a multi-tenant provider network,facilities management systems, stock trading systems, and so on. Sometime series or hierarchies of time series may include very large numbersof measurements. For example, a multi-tenant provider network maymonitor trillions of events per day. As another example, a fulfillmentcenter for an online store may have thousands of sensors that monitorthe state of equipment, goods, and software. In order to efficientlyingest, transform, store, and/or query such large quantities of data,the distributed database 100 may employ scaling techniques while keepingthe database online for continued ingestion and querying. By decouplingvarious stages of the distributed database 100 from each other,individual portions of the database may be scaled up or down by acontrol plane 180 to make better use of computational and storageresources while permitting near-real-time ingestion and querying oftime-series data. Using the techniques described herein, the same orsimilar scaling approaches may be used for time-series metadata (e.g.,schema metadata) as for time-series data.

The ingested time-series data 191 may represent a large number (highcardinality) of individual time series. An individual time series mayinclude a sequence of values or observations (e.g., for a feature of asystem or a phenomenon) that can be plotted over time. An individualtime series may be uniquely identified by a set of dimensions (withdimension values) such as what the observations are measuring, where theobservations were measured, client-specified tags such as device modelor instance type, and so on. For example, a smart-home device mayproduce a time series representing measurements of humidity in aparticular room at a particular address. The same device may alsoproduce other time series representing measurements at the same locationfor temperature, dust levels, carbon dioxide, and so on. As anotherexample, a virtual compute instance in a multi-tenant provider networkmay emit a time series representing CPU utilization over time, anothertime series representing disk reads over time, yet another time seriesrepresenting network packets received over time, and so on. Becausedevelopers often operate on related time series together, time seriesthat are related (e.g., by physical proximity, by being generated by thesame device, and so on) may be clustered using the database 100 forefficient storage and retrieval. To enable such applications, thedatabase 100 may offer a query language that provides filteringaccording to dimensions such as the device model, instance type, region,address, location, and so on, as well as the measure name. In oneembodiment, any change to such a dimension may produce a new time seriesin the database 100.

The database 100 may manage a large amount of time-series datathroughout the lifecycle of the data. The times-series data 191 may bereceived at the database 100 using a fleet of hosts referred to asingestion routers 110. The time-series data may typically arrive at thedatabase 100 in time order, but the database may be able to ingestout-of-order data as well. The ingestion routers 110 may divide the data191 from the clients 190 into non-overlapping ingestion partitions 130.In one embodiment, the ingested data may be spatially partitioned alongnon-overlapping spatial boundaries according to the time series or rangeof the data, one or more tags associated with the data, the region thatproduced the data, the category to which the data belongs, and/or othersuitable metadata. Ingested time-series data may be mapped to differentpartitions based on hierarchical clustering in order to achieve betterperformance of data storage and retrieval. A partition may include onetime series or multiple time series.

A particular time series may be defined by a schema that includescomponents such as one or more dimension names (e.g., “Region”) havingone or more dimension values (e.g., “US-West”) and a measure name (e.g.,“temperature”). As ingested by the time-series database 100, an elementof time-series data may include all the schema components (includingdimension values) as well as a measure value that represents aparticular data point in the time series. In one embodiment, theingestion routers 110 may include a component for schema derivation 111that derives the schema and dimension values for an element oftime-series data from the ingested information and not necessarily froma formal definition of a schema by a customer that manages the clientdevices 190. The time-series database 100 may be referred to as“schema-less” in that a customer need not formally specify the schemausing a control plane 180 or other channel separate from the ingesteddata 191. Thus clients 190 may begin providing the data 191 to thedatabase 100 more quickly and without performing additionalcontrol-plane operations or other configuration tasks.

The ingestion routers 110 may use a clustering scheme 112 to divide theingested data 191 into various partitions 130. The clustering scheme 112may co-locate related time series for optimization of queries and othertasks. The clustering scheme 112 may represent a multi-level hashingscheme in which a hash value for a time series is generated byconcatenating hash values for different components of the schema (anddimension values) for the time series. The clustering scheme 112 may bebased (at least in part) on schemas and dimension values for individualtime series that are derived from ingested data 191 and not necessarilyformally specified by customers. For example, to generate a hashrepresenting an individual time series, a clustering scheme 112 for aparticular table may first hash the measure name for the various timeseries, then concatenate a hash for the dimension names, thenconcatenate a hash for the dimension values. By representing the measurename as a prefix in the hash for an individual time series, various timeseries that have the same measure name may be clustered together in thepartitions 130 and throughout their remaining lifecycle in the database100. As another example, another clustering scheme 112 may first hashthe dimension names, then concatenate a hash for the dimension values,then concatenate a hash for the measure name. As yet another example,another clustering scheme 112 may interleave the dimension names anddimension values (e.g., a hash of a first dimension name, a hash of afirst dimension value, a hash of a second dimension name, a hash of asecond dimension value, and so on) and then concatenate a hash for themeasure name. In one embodiment, the clustering scheme 112 may representa default scheme that is intended to optimize query performance for alarge number (e.g., a majority) of queries for a given table. In oneembodiment, the clustering scheme 112 may be selected by a customer,e.g., on a table-by-table basis.

The ingestion partitions 130 may be maintained using persistent storageresources and may be termed durable partitions. In various embodiments,the durable partitions 130 may be provided by a streaming service orother durable data store 120. The streaming service or durable datastore 120 may also be referred to as a journal. The streaming service ordurable data store 120 may use shards or other divisions to implementthe non-overlapping partitions 130. The use of the durable partitions130 as a staging area may permit the database 100 to decouple ingestionfrom stream processing and storage. Acknowledgements of requests to addtime-series data elements may be sent to the clients 190 upon thesuccessful addition of time-series data elements to the partitions 130.

In addition to the ingestion routers 110, the database 100 may includehosts such as stream processors 140 and query processors 170. A fleet ofstream processors 140 may take the time-series data from the durablepartitions 130, potentially process the data in various ways, and addthe data to one or more storage tiers 150A-150N. For example, one streamprocessor may write data from one partition to a “hot” storage tier, andanother stream processor may write data from the same partition to a“cold” storage tier. In various embodiments, stream processors mayperform reordering, deduplication, aggregation of different timeperiods, rollups, and other transformations on time series data. Streamprocessors 140 may perform tasks such as creating materialized views orderived tables based on a partition, such as an aggregation or rollup ofa time interval. The tasks may include continuous queries that areperformed repeatedly over time, e.g., to create aggregations for eachhour or day of a time series as that time period is finalized. Byco-locating related time-series using the clustering scheme 112, taskssuch as aggregations and cross-series rollups may be optimized orotherwise have their performance improved.

The data 191 may be routed from the durable partitions 130 to the streamprocessors 140 according to routing metadata, e.g., that maps differenttime series or ranges of the data to different stream processors. In oneembodiment, a given stream processor may be assigned to one and only onepartition at a time. In one embodiment, the stream processors 140 mayorganize the time series in tables. The stream processors 140 may alsobe referred to as writers or table builders. A table may store multipletime series. A table may be a named entity that stores related timeseries that are usable by the same application and often managed by thesame customer of the database 100. A data point (e.g., an element) in atime series may be stored in a record. Data points or elements oftime-series data may be added to the database 100 using applicationprogramming interface (API) calls or other programmatic interfaces. Inone embodiment, data points for multiple time series (e.g., for relatedtime series generated by the same client device) with the same timestampmay be added by a client using a single API call. A data point may beassociated with a timestamp, one or more dimensions (in name-valuepairs) representing characteristics of the time series, and a measurerepresenting a variable whose value is tracked over time. Timestamps maybe provided by clients or automatically added upon ingestion. Measuresmay be identified by names and may often have numeric values. Measuresmay be used by the database 100 in generating aggregations such as min,max, average, and count. For example, a time series related toautomobiles may be identified by a unique combination of values fordimensions of a vehicle identification number (VIN), country, state, andcity, while measures for such a time series may include the batterystate and the miles traveled per day. In one embodiment, queries mayspecify time intervals and/or dimension names and/or dimension valuesinstead of or in addition to individual measures.

The database 100 may adapt to varying throughput quickly anddynamically, e.g., such that clients can begin providing time-seriesdata without prior allocation of hosts and storage resources and withoutproviding a formal declaration of a schema. In some embodiments, thecontrol plane 180 may dynamically increase or decrease the number ofpartitions based (at least in part) on the amount or rate of ingestionof time-series data. The database 100 may co-locate the time-series data191 (e.g., the individual measurements or data points) with the schemametadata 192 derived from the ingested data 191 by the ingestion routers110. As shown in FIG. 1, the time-series data 191 may be co-located withthe corresponding schema metadata 192 in the durable partitions 130.Similarly, in the storage tiers such as tier 150A, the time-series data191A may be co-located with the corresponding schema metadata 192A. Thescaling performed by the control plane 180 may thus be applied to boththe time-series data and to the corresponding schema information.

The various storage tiers 150A-150N may represent different use casesfor time-series data. The storage tiers 150A-150N may differ in theirperformance characteristics, durability characteristics, and costcharacteristics. For example, the database 100 may include a hot tier(such as tier 150A) that offers the lowest latency by storing recenttime-series data in volatile memory resources (e.g., random accessmemory) across a distributed set of storages nodes. As another example,the database 100 may include a cold tier that offers higher latency (buta lower cost) by storing a longer interval of time-series data usingpersistent storage resources such as disk drives. The database 100 mayinclude other tiers such as a warm tier that stores recent time-seriesdata in nonvolatile storage resources (e.g., solid-state drives) acrossa distributed set of storages nodes, a frozen tier that stores evenolder time-series data in sequential access storage media, and so on.Based on their needs and budgets, users of the time-series database 100may select and configure one or more of the storage tiers 150A-150N forstorage of their time-series data.

In one embodiment, the database 100 may represent a container of tablesand policies, such as retention policies. Policies may be applied at thedatabase level for all tables or may be overridden for individualtables. The database 100 may offer a control plane 180 that permitscustomers (e.g., developers of applications) and other systems toperform management and modeling of time series data. For example, acomponent for time-series data management of the control plane 180 mayoffer APIs for creating, deleting, and listing tables (or entiredatabases); describing tables and policies; creating and updatingpolicies and associating policies with tables; listing series within atable; and so on. A retention policy may determine the time interval forwhich an element of time-series data is kept in a particular tier;beyond that time interval, the time-series data may expire and may bedeleted from the tier. Different tiers may differ in their retentionpolicies for time-series data. Tables may also differ in their retentionpolicies. In one embodiment, for example, the database 100 may havedefault retention periods of three hours for the hot tier and one yearfor the cold tier. In one embodiment, costs may be assessed to clientsfor the use of the database 100 to store their time-series data, and theper-measure costs assessed for the hot tier may be greater than theper-measure costs for the cold tier. Accordingly, customers may adjustthe retention policies to reach a balance between performance (e.g.,query latency) and cost.

The time-series data may be deemed immutable once written to aparticular storage tier, e.g., such that new values may be appended to atime series but existing values may not be deleted (except forexpiration based on a retention policy). Using a fleet of queryprocessors 170, queries of time-series data may be performed forparticular time intervals. Query processors 170 may perform tasks suchas one-time queries of time-series data in one or more storage tiers150A-150N, transformations of time-series data, and other computations.The database 100 may enable specialized mathematical functions such asinterpolation, approximation, and smoothing to be performed ontime-series data, e.g., in order to find trends and patterns. Bycontrast, traditional relational database management systems may requiredevelopers to write complex application code in order to perform suchfunctions. By interacting with the query processors 170, variousapplications may use the database 100 to perform analysis of time-seriesdata. For example, machine learning and machine vision applications mayuse time-series data managed by the database 100.

Using the clustering scheme 112 based on derived schema metadata 192,various time series that are similar (e.g., that have the same measurename) may be clustered together in the partitions 130 and in the storagetier(s) 150A-150N. Queries for time-series data having the same measurename (or other hash prefix in the selected clustering scheme) may thenbe performed more efficiently. The same clustering scheme 112 may beused throughout the database 100 to optimize both storage and retrievalof time-series data. For example, one of the query processors 170 mayuse the same clustering (hashing) scheme 112 to identify thepartition(s) associated with the hash of the measure name (or otherprefix of the partition hash) and implement the query only using theidentified partition(s) in the relevant storage tier(s).

The time series for a particular customer may have a high cardinality.For example, a set of Internet of Things (IoT) sensor devices 190managed by a customer may emit sensor data over time for a large numberof dimension values (e.g., specific locations), such that eachindividual device and measurement type is represented by a differenttime series (and corresponding schema with dimension values). A customermay seek to explore the time-series schemas under its management (e.g.,in a particular table of the time-series database 100) to generateappropriate queries using some portion of the schemas (and dimensionvalues), perform validation of the schemas, or perform other tasks thatrequire knowledge of the schemas. In one embodiment, the queryprocessors 170 and/or control plane may include a component for schemametadata exploration and acquisition. The schema metadata explorationcomponent may determine all or part of the schema metadata for one ormore time series associated with one or more partitions. For example, auser may use the schema metadata exploration to identify the partitionsfor a particular measure name and then drill down into the dimensionnames and/or dimension values in the schema metadata to build a query.The co-location of time-series data with schema metadata may provide forquick discovery of schemas in order to optimize schema exploration aswell as query performance.

In one embodiment, one or more components of the distributed database100, such as hosts 110, 140 and 170, other compute instances, and/orstorage resources, may be implemented using resources of a providernetwork. The provider network may represent a network set up by anentity such as a private-sector company or a public-sector organizationto provide one or more services (such as various types ofnetwork-accessible computing or storage) accessible via the Internetand/or other networks to a distributed set of clients. The providernetwork may include numerous services that collaborate according to aservice-oriented architecture to provide resources such as the ingestionrouters 110, durable partitions 130, stream processors 140, storageresources 160A-160N, and/or query processors 170. The provider networkmay include numerous data centers hosting various resource pools, suchas collections of physical and/or virtualized computer servers, storagedevices, networking equipment and the like, that are used to implementand distribute the infrastructure and services offered by the provider.Compute resources may be offered by the provider network to clients inunits called “instances,” such as virtual or physical compute instances.In one embodiment, a virtual compute instance may, for example, compriseone or more servers with a specified computational capacity (which maybe specified by indicating the type and number of CPUs, the main memorysize, and so on) and a specified software stack (e.g., a particularversion of an operating system, which may in turn run on top of ahypervisor). In various embodiments, one or more aspects of thedistributed database 100 may be implemented as a service of the providernetwork, the service may be implemented using a plurality of differentinstances that are distributed throughout one or more networks, and eachinstance may offer access to the functionality of the service to variousclients. Because resources of the provider network may be under thecontrol of multiple clients (or tenants) simultaneously, the providernetwork may be said to offer multi-tenancy and may be termed amulti-tenant provider network. In one embodiment, portions of thefunctionality of the provider network, such as the distributed database100, may be offered to clients in exchange for fees.

In one or more of the storage tiers such as tier 150A, the time-seriesdata may be partitioned into a set of tiles along non-overlappingtemporal and spatial boundaries. A tile may thus represent a partitionof time-series data within a time range (between a starting time and anending time) and within a range of keys. The storage resources 160A forsuch a tier 150A may also include a set of storage nodes that aredistributed across various data centers, availability zones, or otherlogical or geographical locations. A tile may be replicated across thestorage nodes with a group of replicas (e.g., three replicas) that areeventually consistent without using a server-side consensus mechanism.

In various embodiments, components of the distributed database 100, suchas the ingestion routers 110, streaming service 120, stream processors140, storage resources 160A-160N, query processors 170, heat analyzer185, and/or control plane 180 may be implemented using any suitablenumber and configuration of computing devices, any of which may beimplemented by the example computing device 3000 illustrated in FIG. 8.In some embodiments, the computing devices may be located in anysuitable number of data centers or geographical locations. In variousembodiments, at least some of the functionality of the distributeddatabase 100 may be provided by the same computing device or bydifferent computing devices. In various embodiments, if any of thecomponents of the distributed database 100 are implemented usingdifferent computing devices, then the components and their respectivecomputing devices may be communicatively coupled, e.g., via one or morenetworks. Any of the components of the distributed database 100 mayrepresent any combination of software and hardware usable to performtheir respective functions. In some embodiments, operations implementedby the distributed database 100 may be performed automatically, e.g.,without a need for user initiation or user intervention after an initialconfiguration stage, and/or programmatically, e.g., by execution ofprogram instructions on at least one computing device. In someembodiments, the distributed database 100 may include additionalcomponents not shown, fewer components than shown, or differentcombinations, configurations, or quantities of the components shown.

Clients 190 of the distributed database 100 may represent externaldevices, systems, or entities with respect to the database. Clientdevices 190 may be managed or owned by one or more customers of thedatabase 100. For example, a particular customer may be a business thatsells sensor devices for installation in residences and businesses, andthose sensor devices may represent the client devices 190. In oneembodiment, the client devices may be implemented using any suitablenumber and configuration of computing devices, any of which may beimplemented by the example computing device 3000 illustrated in FIG. 8.Clients 190 may convey network-based service requests to the ingestionrouter fleet 110 via one or more networks, e.g., to supply a stream ofdata for processing using the stream processors 140 and storage in thestorage tiers 150A-150N. The network(s) may encompass any suitablecombination of networking hardware and protocols necessary to establishnetwork-based communications between client devices 190 and thedistributed database 100. For example, the network(s) may generallyencompass the various telecommunications networks and service providersthat collectively implement the Internet. In one embodiment, thenetwork(s) may also include private networks such as local area networks(LANs) or wide area networks (WANs) as well as public or privatewireless networks. For example, both a given client device and thedistributed database 100 may be respectively provisioned withinenterprises having their own internal networks. In one embodiment, thenetwork(s) may include the hardware (e.g., modems, routers, switches,load balancers, proxy servers, etc.) and software (e.g., protocolstacks, accounting software, firewall/security software, etc.) necessaryto establish a networking link between the given client device and theInternet as well as between the Internet and the distributed database100. In one embodiment, client devices may communicate with thedistributed database 100 using a private network rather than the publicInternet. In various embodiments, the various components of thedistributed database 100 may also communicate with other components ofthe distributed database using one or more network interconnects.

FIG. 2 is a flowchart illustrating a method for schema-based spatialpartitioning in a time-series database, according to some embodiments.As shown in 200, one or more ingestion routers may receive time-seriesdata from clients. A particular time series may be defined by a schemathat includes components such as one or more dimension names (e.g.,“Region”) and a measure name (e.g., “temperature”) and also by dimensionvalues (e.g., “US-West”) for the dimension name(s). As ingested by thetime-series database, an element of time-series data may include all theschema components, one or more dimension values, and a measure valuethat represents a particular data point in the time series. In oneembodiment, the ingestion router(s) may derive the schema for an elementof time-series data from the ingested information and not necessarilyfrom a formal definition of a schema by a customer that manages theclient devices.

As shown in 210, the ingestion router(s) may divide the time-series datainto a plurality of non-overlapping partitions on a spatial dimension.The ingestion router(s) may use a clustering scheme to divide theingested data into the partitions. The clustering scheme may co-locaterelated time series for optimization of queries and other tasks. Theclustering scheme may represent a multi-level hashing scheme in which ahash value for a time series is generated by concatenating hash valuesfor different components of the schema for the time series. Theclustering scheme may be based (at least in part) on schemas anddimension values for individual time series that are derived fromingested data and not necessarily formally specified by customers. Forexample, to generate a hash representing an individual time series, aclustering scheme for a particular table may first hash the measure namefor the various time series, then concatenate a hash for the dimensionnames, then concatenate a hash for the dimension values. By representingthe measure name as a prefix in the hash for an individual time series,various time series that have the same measure name may be clusteredtogether in the partitions and throughout their remaining lifecycle inthe database. In one embodiment, the clustering scheme may represent adefault scheme that is intended to optimize query performance for alarge number (e.g., a majority) of queries for a given table. In oneembodiment, the clustering scheme may be selected by a customer, e.g.,on a table-by-table basis. The schema metadata (including dimensionnames, dimension values, and measure names) may be co-located with thetime-series metadata (e.g., individual measurements or data points) inthe partitions.

As shown in 220, one or more stream processors may store the data fromthe partitions using one or more storage tiers. For example, over time,the stream processor(s) may store time-series data for one partitioninto a series of tiles in a “hot” storage tier and/or a series of filesor other documents in a “cold” storage tier. The schema metadata mayagain be co-located with the time-series metadata (e.g., individualmeasurements or data points) in these tiles or other documents. In oneembodiment, a tile or other partition-specific document in the storagetier(s) may include one or more key-value data stores that represent theschema metadata. These key-value data stores may reflect the particularclustering scheme that is used for the partition. For example, aparticular tile may include a first key-value data store that stores amapping between measure names and dimension names, a second key-valuedata store that stores a mapping between dimension names and dimensionvalues, a third key-value data store that stores a mapping betweendimension values and time-series identifiers, and a fourth key-valuedata store that stores a mapping between time-series identifiers andindividual measurements. The schema metadata (including dimension names,dimension values, and measure names) as well as the correspondingtime-series data (individual measurements) may be encoded using thesekey-value data stores.

As shown in 230, at least some of the schema metadata may be retrievedfor one or more time series in the storage tier(s). In one embodiment,the query processor(s) and/or control plane may include a component forschema metadata exploration or acquisition. The schema metadataexploration component may determine all or part of the schema metadatafor one or more time series associated with one or more partitions.

For example, a user may use the schema metadata exploration to identifythe partitions for a particular measure name and then drill down intothe dimension names in the schema metadata. As another example, a usermay use the schema metadata exploration to identify the partitions for aparticular measure name and dimension name(s) and then drill down intothe dimension values in the schema metadata. The co-location oftime-series data with schema metadata may provide for quick discovery ofschemas in order to optimize schema exploration as well as queries.

As shown in 240, a query may be generated using the retrieved schemametadata. For example, a user who drilled down into the schemas for apartition may build a query that specifies one or more dimension namesand potentially one or more values or ranges of values for the specifieddimension(s). The query may also indicate a time range. As shown in 250,a query processor may perform the query using time-series data in thepartition(s) identified by the schema metadata associated with thequery. In one embodiment, the query processor may use the sameclustering (hashing) scheme to identify the partition(s) associated withthe hash of the measure name (or other prefix of the partition hash) andimplement the query only using the identified partition(s) in therelevant storage tier(s).

FIG. 3A, FIG. 3B, and FIG. 3C examples of hierarchical clustering ofingested time-series data according to derived schemas, according tosome embodiments. The ingestion routers 110 may organize time-seriesdata along a clustering range according to a clustering scheme. Sometime series may be related to other time series via the same dimensionname(s), dimension value(s), and/or measure names. Using schema-basedclustering, related time series may be placed near each other throughouttheir lifecycle in the time-series database 100. The use of schema-basedclustering may achieve a higher degree of compression for time-seriesdata as well as lower latency for queries. The ordering of schemacomponents in the hash built by the clustering scheme may be specifiedby users or may represent a default clustering scheme. Schema components(including dimension names, dimension values, and measure names) may bederived automatically from ingested data. A hash-based clustering schememay be used at various stages of the database 100 to enforce theschema-based clustering. The hash-based clustering scheme may havemultiple levels. The clustering scheme may generate a hash value for atime series by concatenating hash values for different components of theschema for the time series. This hash value may represent a prefix andone or more additional hash components. This hash value may follow atime series throughout its lifecycle in the database 100 such that thesame clustering scheme may be used to identify and organize the timeseries at the stream processors 140, query processors 170, and so on.

As shown in the example of FIG. 3A, a clustering scheme 112A may firsthash the measure name for the various time series, then concatenate ahash for the dimension names, then concatenate a hash for the dimensionvalues. Depending upon the selected or default clustering scheme, thedimension names may be concatenated and then hashed as a unit or hashedseparately and then the individual hashes concatenated. Similarly, thedimension values may be concatenated and then hashed or hashedseparately and then concatenated. By representing the measure name as aprefix in the hash for an individual time series, various time seriesthat have the same measure name may be clustered together in thepartitions 130 and throughout their remaining lifecycle in the database100. For example, one time series may have the measure name 310A“Temperature” as well as the dimension name 320A “DevicePosition” andcorresponding dimension value 325A “9” and the dimension name 330A“Region” and corresponding dimension value 335A “US-West.” Using theclustering scheme 112A that prioritizes the measure name, this timeseries may be clustered with another time series having the measure name310A “Temperature” along with a different dimension name 320B “Location”and corresponding dimension value 325B “PDX.” As another example, yetanother time series may have the measure name 310B “CPU” as well as thedimension name 320A “DevicePosition” and corresponding dimension value325A “9” and the dimension name 330A “Region” and correspondingdimension value 335A “US-West.” Using the clustering scheme 112A thatprioritizes the measure name, this time series may be clustered withanother time series having the measure name 310B “CPU” along with adifferent dimension name 320B “Location” and corresponding dimensionvalue 325B “PDX.”

As shown in the example of FIG. 3B, a clustering scheme 112B may firsthash the dimension names, then concatenate a hash for the measure name,then concatenate a hash for the dimension values. Depending upon theselected or default clustering scheme, the dimension names may beconcatenated and then hashed as a unit or hashed separately and then theindividual hashes concatenated. Similarly, the dimension values may beconcatenated and then hashed or hashed separately and then concatenated.By representing the dimension name(s) as a prefix in the hash for anindividual time series, various time series that have the same dimensionname(s) may be clustered together in the partitions 130 and throughouttheir remaining lifecycle in the database 100. For example, one timeseries may have the dimension name 320A “DevicePosition” andcorresponding dimension value 325A “9” and the dimension name 330A“Region” and corresponding dimension value 335A “US-West,” plus ameasure name 310A “Temperature.” Using the clustering scheme 112A thatprioritizes one or more dimension names, this time series may beclustered with another time series having the same dimension name 320A“DevicePosition” and dimension name 330A “Region” but the differentmeasure name 310B “CPU.” As another example, two additional time serieswith the same dimension name 320B “Location” may be clustered togethereven though they have different measure names 310A “Temperature” and310B “CPU.”

As shown in the example of FIG. 3C, a clustering scheme 112C mayinterleave the dimension names and dimension values (e.g., a hash of afirst dimension name, a hash of a first dimension value, a hash of asecond dimension name, a hash of a second dimension value, and so on)and then concatenate a hash for the measure name. Using a clusteringscheme 112C with interleaving of dimension names and dimension values,various time series that have the same dimension name(s) and value(s)may be clustered together in the partitions 130 and throughout theirremaining lifecycle in the database 100. For example, one time seriesmay have the dimension name 320A “DevicePosition” and correspondingdimension value 325A “9” and the dimension name 330A “Region” andcorresponding dimension value 335A “US-West,” plus a measure name 310A“Temperature.” Using the clustering scheme 112A that prioritizes one ormore dimension names and their values, this time series may be clusteredwith another time series having the same dimension name 320A“DevicePosition” and same value 325A and the same dimension name 330A“Region” and same value 335A but the different measure name 310B “CPU.”As another example, two additional time series with the same dimensionname 320B “Location” and value 325B “PDX” but different measure names310A “Temperature” and 310B “CPU” may be clustered together.

Data points for the time series shown in FIG. 3A, FIG. 3B, and FIG. 3Cmay be mapped to various durable partitions by the ingestion routers110. As shown in the example, the time-series data may be mapped androuted to partitions 130A and 130B. In one embodiment, different numbersof time series may be mapped to different partitions based (at least inpart) on the ingestion rate of those time series. Partitions may besplit or merged as appropriate to adapt to changing ingestion rates forvarious time series. Each durable partition may support streaming. Aparticular partition may be mapped to a particular stream processor,e.g., for writing data from the partition to a particular storage tier.In one embodiment, partitions 130A-130B may represent shards of adurable data store or streaming service 120. In one embodiment,partitions 130A-130B may represent database tables or other durablestorage resources.

FIG. 4 illustrates an example of storage of time-series data usingtwo-dimensional tiles in a hot tier, according to some embodiments. Asdiscussed above, the database 100 may include a hot storage tier such astier 150A that stores recent data with high availability and lowlatency. In one embodiment, the hot tier 150A may include a set ofstorage hosts or storage nodes that include computational resources andmemory resources. The storage nodes may store time-series data usingtiles that are generated or appended to by stream processors. Tiles maybe stored using storage resources such as memory (e.g., RAM) and/orsolid-state drives for lower latency of storage and retrieval. Tiles maybe replicated across different nodes (e.g., in different data centers oravailability zones) for improved durability. Tiles may be partitionedalong non-overlapping spatial boundaries, e.g., such that time-seriesdata from one time series is assigned to one tile while time-series datafrom another time series is assigned to another tile. However, a tilemay hold one or more time series. The spatial range may be based onschema-based clustering that seeks to co-locate related time series inthe same partition, and the schema-based clustering may be performed bythe ingestion routers 110. Tiles may also be partitioned alongnon-overlapping temporal boundaries. Due to the spatial dimension 401and the temporal dimension 409, tiles may be said to be two-dimensional.The two-dimensional partitioning represented in tiles may be decoupledfrom the partitioning of the ingestion stage due to the difference inwrite latency between the stages. The same partitioning scheme may beused, but the partition ranges may differ. In one embodiment, if theclustering scheme is changed, then subsequent tiles may be reorganizedto reflect the clustering change over time.

In the example of FIG. 4, a set of time series may be mapped to durablepartitions 130A, 130B, and 130C based on a spatial range (e.g.,schema-based clustering). Particular partitions may be mapped toparticular stream processors for writing data from the partitions to thehot tier 150A. For example, partition 130A may be assigned to streamprocessor 140A that writes to the hot tier, partition 130B may beassigned to stream processor 140B that writes to the hot tier, andpartition 130C may be assigned to stream processor 140C that writes tothe hot tier. For a given time series or partition, tiles representingolder windows of time may be termed “closed,” while a tile representinga current window of time may be termed “open.” Tiles may be closed whenthe amount of data reached a threshold or when a maximum time intervalis reached. For current data points (e.g., data not received out oforder), the stream processor for a partition may write to an open tile.Out-of-order data may be routed to previously closed tiles in somecircumstances. Tiles whose temporal boundaries are beyond the retentionperiod (e.g., three hours) for the tier and table may be deemed expiredand either deleted or marked for deletion. As shown in the example ofFIG. 4, stream processor 140A may write to an open tile 410A3 that waspreceded in time by a tile 410A2 that was preceded in time by anow-expired tile 410A. Similarly, stream processor 140B may write to anopen tile 410B4 that was preceded in time by a tile 410B3 that waspreceded in time by a tile 410B2 that was preceded in time by anow-expired tile 410B1. Additionally, stream processor 140C may write toan open tile 410C2 that was preceded in time by a tile 410C1. Asdiscussed above, the contents of a tile may be replicated (e.g., usingthree replicas) across different location or zones to achieve greaterdurability of the hot tier.

FIG. 5 illustrates the storage of time-series data along with schemametadata using a set of key-value data stores, according to someembodiments. In one embodiment, a tile or other partition-specificdocument in the storage tier(s) may include one or more key-value datastores that represent the schema metadata and that point to one another.These key-value data stores may reflect the particular clustering schemethat is used for the partition. For example, a particular tile 410A3 mayinclude a first key-value data store 510A that stores a mapping betweenmeasure names and dimension names, a second key-value data store 520Athat stores a mapping between dimension names and dimension values, athird key-value data store 530A that stores a mapping between dimensionvalues and time-series identifiers, and a fourth key-value data store540A that stores a mapping between time-series identifiers andindividual measurements. Similarly, a particular tile 410B4 may includea first key-value data store 510B that stores a mapping between measurenames and dimension names, a second key-value data store 520B thatstores a mapping between dimension names and dimension values, a thirdkey-value data store 530B that stores a mapping between dimension valuesand time-series identifiers, and a fourth key-value data store 540B thatstores a mapping between time-series identifiers and individualmeasurements. The schema metadata as well as the time-series data(individual measurements) may be encoded using these key-value datastores. The co-located and clustered schema metadata may be used forvalidation across different time series. For example, a customer may usethe schema metadata to ensure that the total number of schemas for atable is less than some threshold amount (e.g., one billion).

FIG. 6 illustrates the querying of time-series data using schemametadata co-located with the time-series data, according to someembodiments. In one embodiment, a query processor 170A may include acomponent 171 for schema metadata acquisition. The schema metadataacquisition component 171 may determine all or part of the schemametadata for one or more time series associated with one or morepartitions. For example, a user may use the schema metadata acquisition171 to identify the partitions for a particular measure name and thendrill down into the dimension names in the schema metadata, e.g., intiles 410A3 and/or 410B4 in the storage tier 150A. The co-location oftime-series data with schema metadata using key-value data stores to mapone portion of the schema to another may provide for quick discovery ofschemas in order to optimize schema exploration as well as queries.

In one embodiment, the query processor 170A may include a component 172for query generation. Using this component 172, a query may be generatedusing the acquired schema metadata. For example, a user who drilled downinto the schemas for a partition may build a query that specifies one ormore dimension names and potentially one or more values or ranges ofvalues for the specified dimension(s). The query may also indicate atime range. In one embodiment, the query processor 170A may include acomponent 173 for query implementation. Using this component 173, thequery processor 170A may perform the query using time-series data in thepartition(s) identified by the schema metadata associated with thequery. In one embodiment, the query processor 170A may use the sameclustering (hashing) scheme 112 to identify the partition(s) associatedwith the hash of the measure name (or other prefix of the partitionhash) and implement the query only using the identified partition(s) inthe relevant storage tier(s).

FIG. 7 illustrates further aspects of the example system environment forschema-based spatial partitioning in a time-series database, includingthe use of a user-specified ordering of schema components forclustering, according to some embodiments. In one embodiment, customersmay indicate orderings of schema components for particular time series,e.g., to optimize storage and retrieval of time-series data for theneeds of particular customers. A shown in FIG. 7, a customer may providea user-specified ordering 780. The ordering 780 may represent a relativepriority of all or part of the schema components (dimension name(s),dimension value(s), and measure name). For example, the ordering 780 mayindicate that the measure name should be hashed first, and then adefault ordering should be used (e.g., dimension name(s) and thendimension value(s)). As another example, the ordering 780 may indicatethat a particular dimension name (e.g., “Region”) should be hashed firstand another dimension name (e.g., “DeviceType”) should be hashed next.In the absence of such customer-provided information, a defaultclustering scheme may be used to partition time-series data such thatthe majority of queries are optimized.

Illustrative Computer System

In at least some embodiments, a computer system that implements aportion or all of one or more of the technologies described herein mayinclude a computer system that includes or is configured to access oneor more computer-readable media. FIG. 8 illustrates such a computingdevice 3000 according to one embodiment. In the illustrated embodiment,computing device 3000 includes one or more processors 3010A-3010Ncoupled to a system memory 3020 via an input/output (I/O) interface3030. In one embodiment, computing device 3000 further includes anetwork interface 3040 coupled to I/O interface 3030.

In various embodiments, computing device 3000 may be a uniprocessorsystem including one processor or a multiprocessor system includingseveral processors 3010A-3010N (e.g., two, four, eight, or anothersuitable number). In one embodiment, processors 3010A-3010N may includeany suitable processors capable of executing instructions. For example,in various embodiments, processors 3010A-3010N may be processorsimplementing any of a variety of instruction set architectures (ISAs),such as the x86, PowerPC, SPARC, or MIPS ISAs, or any other suitableISA. In one embodiment, in multiprocessor systems, each of processors3010A-3010N may commonly, but not necessarily, implement the same ISA.

In one embodiment, system memory 3020 may be configured to store programinstructions and data accessible by processor(s) 3010A-3010N. In variousembodiments, system memory 3020 may be implemented using any suitablememory technology, such as static random access memory (SRAM),synchronous dynamic RAM (SDRAM), nonvolatile/Flash-type memory, or anyother type of memory. In the illustrated embodiment, programinstructions and data implementing one or more desired functions, suchas those methods, techniques, and data described above, are shown storedwithin system memory 3020 as code (i.e., program instructions) 3025 anddata 3026.

In one embodiment, I/O interface 3030 may be configured to coordinateI/O traffic between processors 3010A-3010N, system memory 3020, and anyperipheral devices in the device, including network interface 3040 orother peripheral interfaces. In some embodiments, I/O interface 3030 mayperform any necessary protocol, timing or other data transformations toconvert data signals from one component (e.g., system memory 3020) intoa format suitable for use by another component (e.g., processors3010A-3010N). In some embodiments, I/O interface 3030 may includesupport for devices attached through various types of peripheral buses,such as a variant of the Peripheral Component Interconnect (PCI) busstandard or the Universal Serial Bus (USB) standard, for example. Insome embodiments, the function of I/O interface 3030 may be split intotwo or more separate components, such as a north bridge and a southbridge, for example. In some embodiments, some or all of thefunctionality of I/O interface 3030, such as an interface to systemmemory 3020, may be incorporated directly into processors 3010A-3010N.

In one embodiment, network interface 3040 may be configured to allowdata to be exchanged between computing device 3000 and other devices3060 attached to a network or networks 3050. In various embodiments,network interface 3040 may support communication via any suitable wiredor wireless general data networks, such as types of Ethernet network,for example. Additionally, in some embodiments, network interface 3040may support communication via telecommunications/telephony networks suchas analog voice networks or digital fiber communications networks, viastorage area networks such as Fibre Channel SANs, or via any othersuitable type of network and/or protocol.

In some embodiments, system memory 3020 may be one embodiment of acomputer-readable (i.e., computer-accessible) medium configured to storeprogram instructions and data as described above for implementingembodiments of the corresponding methods and apparatus. In someembodiments, program instructions and/or data may be received, sent orstored upon different types of computer-readable media. In someembodiments, a computer-readable medium may include non-transitorystorage media or memory media such as magnetic or optical media, e.g.,disk or

DVD/CD coupled to computing device 3000 via I/O interface 3030. In oneembodiment, a non-transitory computer-readable storage medium may alsoinclude any volatile or nonvolatile media such as RAM (e.g. SDRAM, DDRSDRAM, RDRAM, SRAM, etc.), ROM, etc., that may be included in someembodiments of computing device 3000 as system memory 3020 or anothertype of memory. In one embodiment, a computer-readable medium mayinclude transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as a network and/or a wireless link, such as may be implemented vianetwork interface 3040. The described functionality may be implementedusing one or more non-transitory computer-readable storage media storingprogram instructions that are executed on or across one or moreprocessors. Portions or all of multiple computing devices such as thatillustrated in FIG. 8 may be used to implement the describedfunctionality in various embodiments; for example, software componentsrunning on a variety of different devices and servers may collaborate toprovide the functionality in one embodiment. In some embodiments,portions of the described functionality may be implemented using storagedevices, network devices, or various types of computer systems. Invarious embodiments, the term “computing device,” as used herein, refersto at least all these types of devices, and is not limited to thesetypes of devices.

The various methods as illustrated in the Figures and described hereinrepresent examples of embodiments of methods. In various embodiments,the methods may be implemented in software, hardware, or a combinationthereof. In various embodiments, in various ones of the methods, theorder of the steps may be changed, and various elements may be added,reordered, combined, omitted, modified, etc. In various embodiments,various ones of the steps may be performed automatically (e.g., withoutbeing directly prompted by user input) and/or programmatically (e.g.,according to program instructions).

The terminology used in the description of the invention herein is forthe purpose of describing particular embodiments only and is notintended to be limiting of the invention. As used in the description ofthe invention and the appended claims, the singular forms “a”, “an” and“the” are intended to include the plural forms as well, unless thecontext clearly indicates otherwise. It will also be understood that theterm “and/or” as used herein refers to and encompasses any and allpossible combinations of one or more of the associated listed items. Itwill be further understood that the terms “includes,” “including,”“comprises,” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

As used herein, the term “if” may be construed to mean “when” or “upon”or “in response to determining” or “in response to detecting,” dependingon the context. Similarly, the phrase “if it is determined” or “if [astated condition or event] is detected” may be construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event],” depending on the context.

It will also be understood that, although the terms first, second, etc.,may be used herein to describe various elements, these elements shouldnot be limited by these terms. These terms are only used to distinguishone element from another. For example, a first contact could be termed asecond contact, and, similarly, a second contact could be termed a firstcontact, without departing from the scope of the present invention. Thefirst contact and the second contact are both contacts, but they are notthe same contact.

Numerous specific details are set forth herein to provide a thoroughunderstanding of claimed subject matter. However, it will be understoodby those skilled in the art that claimed subject matter may be practicedwithout these specific details. In other instances, methods, apparatus,or systems that would be known by one of ordinary skill have not beendescribed in detail so as not to obscure claimed subject matter. Variousmodifications and changes may be made as would be obvious to a personskilled in the art having the benefit of this disclosure. It is intendedto embrace all such modifications and changes and, accordingly, theabove description is to be regarded in an illustrative rather than arestrictive sense.

What is claimed is:
 1. A system, comprising: one or more ingestionrouters of a time-series database, wherein the one or more ingestionrouters are configured to: receive elements of time-series data of aplurality of time series from a plurality of clients, wherein the timeseries are associated with respective schemas, wherein an individualschema comprises one or more dimension names and a measure name; anddivide the elements of time-series data into a plurality ofnon-overlapping partitions according to a clustering scheme, wherein theclustering scheme represents a multi-level hashing scheme, wherein theclustering scheme is determined based at least in part on the schemasand at least in part on a plurality of dimension values, and wherein theschemas are determined based at least in part on the elements oftime-series data; one or more stream processors of the time-seriesdatabase, wherein the one or more stream processors are configured to:store, into one or more storage tiers, the elements of time-series datafrom the plurality of non-overlapping partitions; and one or more queryprocessors of the time-series database, wherein the one or more queryprocessors are configured to: perform a query of the time-series data inone or more of the storage tiers, wherein the query is performed basedat least in part on the clustering scheme.
 2. The system as recited inclaim 1, wherein the clustering scheme comprises clustering theplurality of time series first by measure names, then by dimensionnames, and then by dimension values.
 3. The system as recited in claim2, wherein the query indicates a particular measure name, and wherein,in performing the query, the one or more query processors are furtherconfigured to: determine a hash of the particular measure name;determine a particular one or more of the non-overlapping partitionsthat are associated with the hash of the particular measure name; anddetermine a result of the query using the particular one or more of thenon-overlapping partitions that are associated with the hash of theparticular measure name.
 4. The system as recited in claim 2, whereinthe plurality of non-overlapping partitions comprise key-value datastores that store a mapping of the measure names to the dimension names,a mapping of the dimension names to the dimension values, and a mappingof the dimension values to time-series identifiers, and wherein thequery is determined based at least in part on the one or more of theschemas using the key-value data stores.
 5. A method, comprising:dividing, by a time-series database comprising a plurality of hosts,elements of time-series data of a plurality of time series into aplurality of partitions according to a clustering scheme, wherein thetime series are associated with respective schemas, wherein the schemascomprise a plurality of dimension names and a plurality of measurenames, wherein the clustering scheme is determined based at least inpart on the schemas and at least in part on a plurality of dimensionvalues, and wherein the schemas are determined based at least in part onthe elements of time-series data; storing, by the time-series database,the elements of time-series data from the plurality of partitions intoone or more storage tiers; and performing, by the time-series database,a query of the time-series data in one or more of the storage tiers,wherein the query is performed based at least in part on the clusteringscheme.
 6. The method as recited in claim 5, wherein the clusteringscheme comprises clustering the plurality of time series first by themeasure names.
 7. The method as recited in claim 5, wherein theclustering scheme comprises clustering the plurality of time seriesfirst by the measure names, then by the dimension names, and then by thedimension values.
 8. The method as recited in claim 7, wherein the queryindicates a particular measure name, and wherein the method furthercomprises: determining a hash of the particular measure name;determining a particular one or more of the partitions that areassociated with the hash of the particular measure name; and determininga result of the query using elements of time-series data from theparticular one or more of the partitions that are associated with thehash of the particular measure name.
 9. The method as recited in claim7, wherein the plurality of partitions comprise key-value data storesthat store a mapping of the measure names to the dimension names, amapping of the dimension names to the dimension values, and a mapping ofthe dimension values to time-series identifiers, and wherein the queryis determined based at least in part on the one or more of the schemasusing the key-value data stores.
 10. The method as recited in claim 7,further comprising: determining a hash of the particular measure name;determining a particular one or more of the partitions that areassociated with the hash of the particular measure name; and determiningone or more additional portions of a plurality of schemas having theparticular measure name using the particular one or more of thepartitions that are associated with the hash of the particular measurename.
 11. The method as recited in claim 5, wherein the clusteringscheme is determined based at least in part on input indicating anordering among one or more dimension names, one or more dimensionvalues, and a measure name for an individual schema.
 12. The method asrecited in claim 5, further comprising: performing, by the time-seriesdatabase, validation of one or more of the schemas having a particularmeasure name, wherein the validation is performed using one or more ofthe partitions associated with the particular measure name.
 13. Themethod as recited in claim 5, further comprising: ingesting, by thetime-series database, additional elements of the time-series data havinga modified schema; and modifying a partitioning of the time-seriesdatabase based at least in part on the modified schema.
 14. The methodas recited in claim 5, wherein the clustering scheme comprisesinterleaving at least some of the dimension names and at least some ofthe dimension values.
 15. One or more non-transitory computer-readablestorage media storing program instructions that, when executed on oracross one or more processors, perform: dividing, by one or moreingestion routers of a time-series database, elements of time-seriesdata of a plurality of time series into a plurality of partitionsaccording to a multi-level hashing scheme, wherein the time series areassociated with respective schemas, wherein the schemas comprise aplurality of dimension names and a plurality of measure names, whereinthe multi-level hashing scheme is determined based at least in part onthe schemas and at least in part on a plurality of dimension values, andwherein the schemas are determined based at least in part on theelements of time-series data; storing, by one or more stream processorsof the time-series database, the elements of time-series data from theplurality of partitions into one or more storage tiers; and performing,by one or more query processors of the time-series database, a query ofthe time-series data in one or more of the storage tiers, wherein thequery is performed based at least in part on the multi-level hashingscheme.
 16. The one or more non-transitory computer-readable storagemedia as recited in claim 15, wherein the multi-level hashing schemecomprises clustering the plurality of time series first by the measurenames and next by the dimension names and the dimension values.
 17. Theone or more non-transitory computer-readable storage media as recited inclaim 16, further comprising additional program instructions that, whenexecuted on or across the one or more processors, perform: determining ahash of a particular measure name and one or more particular dimensionnames indicated in the query; determining a particular one or more ofthe partitions that are associated with the hash of the particularmeasure name and the one or more particular dimension names; anddetermining a result of the query using elements of time-series datafrom the particular one or more of the partitions that are associatedwith the hash of the particular measure name and the one or moreparticular dimension names.
 18. The one or more non-transitorycomputer-readable storage media as recited in claim 16, wherein theplurality of partitions comprise a plurality of key-value data storesthat store a mapping of the measure names to the dimension names, amapping of the dimension names to the dimension values, and a mapping ofthe dimension values to time-series identifiers, and wherein the queryis determined based at least in part on the one or more of the schemasusing the key-value data stores.
 19. The one or more non-transitorycomputer-readable storage media as recited in claim 15, wherein themulti-level hashing scheme is determined based at least in part on userinput indicating an ordering among one or more dimension names, one ormore dimension values, and a measure name for an individual schema. 20.The one or more non-transitory computer-readable storage media asrecited in claim 15, wherein the multi-level hashing scheme comprisesinterleaving at least some of the dimension names and at least some ofthe dimension values.